Information processing device, information processing method, and program

ABSTRACT

[Object] To provide an information processing device, an information processing method, and a program that can enable more comfortable communication by rendering distances to multiple communication partners by using three-dimensional expression for connection between spaces. 
     [Solution] An information processing device including: a reception unit configured to receive data from a communication destination; a generation unit configured to generate spatial image information placed at a predetermined position in a three-dimensional space by using captured images of a plurality of the communication destinations; and a display control unit configured to perform control so as to display the generated spatial image information on a display unit.

TECHNICAL FIELD

The present disclosure relates to an information processing device, aninformation processing method, and a program.

BACKGROUND ART

In recent years, as communication technologies have developed, TV phoneconference systems for having a conversation via video at remotelocations have been proposed. Specifically, for example, a displaydevice, a camera, a microphone, and a speaker are provided in eachspace, and captured video and collected audio data in a certain spaceare output in real time from a display device and a speaker installed inanother space.

In regard to such two-way communication technologies, Patent Literature1 below, for example, proposes a technology of, when performing two-waycommunication using a window-like imaging/display device, making itpossible to see video corresponding to a change in position and angle ofa person on the viewing side.

In addition, Patent Literature 2 below proposes determining a degree ofrequest for communication of oneself and a communication partner on thebasis of user state information, which makes it possible to performcomfortable communication convenient for each other.

CITATION LIST Patent Literature

Patent Literature 1: JP 4576740B

Patent Literature 2: JP 4645355B

DISCLOSURE OF INVENTION Technical Problem

However, Patent Literature 1 described above has an effect of makingremote spaces feel as if being connected through one window by using thewindow-like imaging/display device, but may cause reluctance to thepartner being in the immediate neighborhood, because the spaces are incontact at zero distance.

In addition, Patent Literature 2 described above copes with timing ofconnecting spaces (i.e., calling timing), but does not refer to adistance between spaces.

Hence, the present disclosure proposes an information processing device,an information processing method, and a program that can enable morecomfortable communication by rendering distances to multiplecommunication partners by using three-dimensional expression forconnection between spaces.

Solution to Problem

According to the present disclosure, there is proposed an informationprocessing device including: a reception unit configured to receive datafrom a communication destination; a generation unit configured togenerate spatial image information placed at a predetermined position ina three-dimensional space by using captured images of a plurality of thecommunication destinations; and a display control unit configured toperform control so as to display the generated spatial image informationon a display unit.

According to the present disclosure, there is proposed an informationprocessing method including, by a processor: receiving data from acommunication destination; generating spatial image information placedat a predetermined position in a three-dimensional space by usingcaptured images of a plurality of the communication destinations; andperforming control so as to display the generated spatial imageinformation on a display unit.

According to the present disclosure, there is proposed a program causinga computer to function as: a reception unit configured to receive datafrom a communication destination; a generation unit configured togenerate spatial image information placed at a predetermined position ina three-dimensional space by using captured images of a plurality of thecommunication destinations; and a display control unit configured toperform control so as to display the generated spatial image informationon a display unit.

Advantageous Effects of Invention

According to the present disclosure as described above, more comfortablecommunication can be enabled by rendering distances to multiplecommunication partners by using three-dimensional expression forconnection between spaces.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates an overview of a communication system according to anembodiment of the present disclosure.

FIG. 2 is a diagram for describing a problem in display in connectionwith multiple locations.

FIG. 3 is a diagram for describing a display example of video ofmultiple locations according to an embodiment of the present disclosure.

FIG. 4 is a block diagram illustrating an example of a configuration ofan information processing device according to the present embodiment.

FIG. 5 is a flowchart for describing communication control processingaccording to the present embodiment.

FIG. 6 is a flowchart illustrating communication state visualizationprocessing according to the present embodiment.

FIG. 7 illustrates a display example of a 3D space in which houseobjects are arranged at a medium distance according to the presentembodiment.

FIG. 8 illustrates an example of privacy protection according to thepresent embodiment.

FIG. 9 illustrates a display example of a 3D space in which houseobjects are arranged at a short distance according to the presentembodiment.

FIG. 10 illustrates an example of zero-distance display according to thepresent embodiment.

FIG. 11 illustrates a display example of a 3D space in which houseobjects are arranged at a long distance according to the presentembodiment.

FIG. 12 illustrates an example in which occurrence of communication isvisualized according to the present embodiment.

FIG. 13 is a diagram for describing a type of visualization expressionaccording to the present embodiment.

FIG. 14 is a diagram for describing another type of visualizationexpression according to the present embodiment.

FIG. 15 is a diagram for describing another type of visualizationexpression according to the present embodiment.

FIG. 16 is a diagram for describing a method for expressing activity ofconversation by intensity of a waveform according to the presentembodiment.

FIG. 17 is a diagram for describing a method for expressing activity ofconversation by the size and number of speech balloons according to thepresent embodiment.

FIG. 18 is a diagram for describing a method for expressing activity ofconversation by darkness of color of display connecting virtual windowsaccording to the present embodiment.

FIG. 19 is a diagram for describing a method for expressing activity ofconversation by density of a broken line according to the presentembodiment.

FIG. 20 is a diagram for describing a method for expressing activity ofconversation by strength of light around virtual windows according tothe present embodiment.

FIG. 21 is a diagram for describing a method for expressing a mood ofconversation by a broken line according to the present embodiment.

FIG. 22 is a diagram for describing a method for expressing a mood ofconversation by speech balloons according to the present embodiment.

FIG. 23 illustrates an example of participation rejection displayaccording to the present embodiment.

FIG. 24 illustrates another example of participation rejection displayaccording to the present embodiment.

FIG. 25 illustrates another example of participation rejection displayaccording to the present embodiment.

FIG. 26 illustrates an example of participation request displayaccording to the present embodiment.

FIG. 27 illustrates another example of participation request displayaccording to the present embodiment.

FIG. 28 is an explanatory diagram illustrating a hardware configurationof an information processing device according to the present disclosure.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

In addition, description will be given in the following order.

1. Overview of communication system according to embodiment of presentdisclosure2. Configuration of information processing device3. Operation processing4. Display examples of spatial information4-1. Medium-distance display4-2. Short-distance display4-3. Zero-distance display4-4. Long-distance display4-5. Visualization of communication state(4-5-1. Types of visualization expression)(4-5-2. Expression of activity of conversation)(4-5-3. Expression of mood of conversation)(4-5-4. Participation rejection display)(4-5-5. Participation request display)5. Hardware configuration

6. Conclusion 1. Overview of Information Processing Device According toEmbodiment of Present Disclosure

An overview of a communication system 1 according to an embodiment ofthe present disclosure is described with reference to FIG. 1. FIG. 1illustrates an overall configuration of the communication system 1according to an embodiment of the present disclosure.

As illustrated in FIG. 1, in the communication system 1 according to thepresent embodiment, information processing devices 10A, 10B, and 10C areconnected via a network 20. The information processing devices 10A, 10B,and 10C are placed respectively in space A, space B, and space C, andthe spaces are connected by an information channel of video, sound, andthe like, which makes it possible to feel as if the spaces areconnected. For example, when a parent and a child live away from eachother, their living spaces can be made to look connected to each othervia a display device corresponding to a window or a door. In the casewhere living spaces are connected to each other, a user can live whileroughly grasping circumstances of a partner space (e.g., circumstancesof a child, circumstances of a parent, or the like).

The information processing devices 10A to 10C perform synchronizationprocessing of two-way communication, calculation and control of spatialdistance, etc.; the calculation and control of separation distance, etc.may be performed by a processing server 30. In addition, thecommunication system 1 includes the processing server 30 in the exampleillustrated in FIG. 1, but the present disclosure is not limited tothis, and the processing server 30 does not need to be included.

BACKGROUND

Here, in general video chat technologies, connection with acommunication partner is controlled by switching between 0 and 1, suchas display/non-display of video or ON/OFF of sound, and the degree ofconnection cannot be adjusted finely. Consequently, a high sense ofrealism constantly occurs, which cannot meet the needs of users such aswanting to keep feeling the partner's circumstances. None of the abovepatent literatures describe controlling a distance between connectedspaces.

In addition, in a virtual living-together experience system using alarge-screen device like a window, in a case of one-to-one (one locationto one location) connection with a partner, simple control of showingone video is possible; however, in a case of connection with multiplespaces, there are problems in that a sense of scale is lost by simplescreen division, and the strength of communication with each partnerbecomes difficult to adjust.

For example, as illustrated in FIG. 2, when video of multiplecommunication destinations (video 81 of space B and video 82 of space C)is displayed by simple screen division in a large-screen device 80 likea window installed on a wall or the like, a sense of life-size and asense of scale are lost, and virtual living-together experience cannotbe sufficiently expressed.

Hence, the present embodiment uses three-dimensional expression forconnection between spaces to enable more comfortable communicationwithout loss of a sense of life-size and a sense of scale. A specificdisplay example is described with reference to FIG. 3. FIG. 3 is adiagram for describing a display example of video of multiple locationsaccording to an embodiment of the present disclosure. As illustrated inFIG. 3, for example, a three-dimensional space is displayed on a display1091, and virtual house objects 21B and 21C corresponding to respectivespaces are arranged in the three-dimensional space. The display 1091 isa large-screen display device, and is installed on a wall or the like,being likened to a window or a doorway. Here, the display example isdescribed using the display 1091 installed in space A, for example. Notethat although not illustrated in FIG. 3, a camera 1011, a mic 1012, asensor 1013, a speaker 1092, and an indicator 1093 (LED etc.) are alsoinstalled in space A, such as around the display 1091. There is noparticular limitation on where and how many of them are installed.

The virtual house objects 21B and 21C arranged in the three-dimensionalspace respectively correspond to communication destination spaces B andC, and video of each communication destination is fitted into a windowor a doorway. That is, video 22B fitted into a window of the houseobject 21B is video captured in space B, and video 22C fitted into thehouse object 21C is video captured in space C. This makes it possible tograsp circumstances of the partner's house through the window of eachother's house even in a state where the houses are slightly away. Sincea distance to the partner's house is somewhat kept, constant connectiondoes not cause a high sense of realism, and virtual living-togetherexperience as if staying in the same space can be obtained. In addition,sound may be made to be heard in a volume corresponding to the distanceto the partner's house, for example. In addition, in accordance with aposition of each house in the three-dimensional space, auditorylateralization may be performed on sound from each house according tostereo sound. Note that, as an example here, a house object is used as avirtual object into which video of a communication destination isfitted. However, the present embodiment is not limited to this; forexample, a virtual object such as a vehicle (e.g., a car or a train), abuilding other than a house, a structure, or a natural object may beused, and video of a communication destination may be fitted into anopening such as a window or a door.

In addition, as illustrated in FIG. 3, a stage 24 may be provided nearthe center of video of a three-dimensional space, which enablescommunication to be performed via a virtual object on the stage 24 in avirtual space. A shape and a position of the stage 24 are notparticularly limited, and for example, a range surrounded by houses maybe used as a virtual courtyard space instead of the stage 24.

Then, the communication system 1 according to the present embodimentmakes a distance between houses in a three-dimensional space displayedon the display 1091 longer and shorter, thereby controlling a degree ofconnection between connected spaces, and can enable more comfortablecommunication without loss of a sense of life-size and a sense of scale.

The overview of the communication system 1 according to an embodiment ofthe present disclosure has been described. Now, a configuration of theinformation processing device 10 used in the communication system 1 willbe described with reference to FIG. 4.

2. CONFIGURATION OF INFORMATION PROCESSING DEVICE

FIG. 4 is a block diagram illustrating an example of a configuration ofthe information processing device 10 according to the presentembodiment. As illustrated in FIG. 4, the information processing device10 includes an input unit 101, a spatial information acquisition unit102, a state determination unit 103, a spatial distance control unit104, an operation interface (I/F) 105, a 3D space generation unit 106, acommunication unit 107, a spatial information generation unit 108, anoutput unit 109, and a storage unit 110.

The input unit 101 has a function of inputting spatial information. Forexample, the input unit 101 is implemented by the camera 1011, the mic(an abbreviation of microphone) 1012, and the sensor 1013. The camera1011, which may include a plurality of cameras, images the inside of aspace (e.g., a living room) to acquire a captured image. In addition,the mic 1012, which may include a plurality of mics, collects sound in aspace and environmental sound around the space (e.g., in a neighboringroom, a hall, the outside of a house, etc.) to acquire audio data. Inaddition, the sensor 1013 has a function of detecting various types ofinformation in a space or around the space, and a temperature sensor, ahumidity sensor, an illuminance sensor, a motion detector, a dooropening/closing sensor, and the like are assumed as examples.

The spatial information acquisition unit 102 acquires various types ofspatial information from the input unit 101, prepares data so that thedata can be used as a material for state determination by the statedetermination unit 103, and outputs the data. Preparation of data is,for example, noise processing, image analysis, object recognition, voiceanalysis, or the like. Furthermore, the spatial information acquisitionunit 102 performs user recognition on the basis of the acquired spatialinformation. As user recognition, recognition of the following isassumed: personal identification of a user in the space, and a position(where the user is in the room, etc.), a posture (whether the user isstanding, sitting, or sleeping, etc.), an emotion (whether the user ishaving fun or being sad, etc.), an action (preparing for dinner,watching TV, reading a book, etc.), and busyness (whether the user isbusily moving around, etc.) of the user. In addition, the spatialinformation acquisition unit 102 performs environment recognition on thebasis of the acquired spatial information. As environment recognition,recognition of the following is assumed: the current time slot (morning,afternoon, evening, midnight), brightness (brightness of the room, lightfrom a window), temperature, sound (sound collected in the space), aregion (a place where the space is present), unclutteredness (how muchthe room is tidied up), and the like of the space.

The state determination unit 103 determines a state of a space or astate of a user (i.e., context of a communication source space) on thebasis of information acquired and output by the spatial informationacquisition unit 102. For example, the state determination unit 103determines the state of the space or the state of the user on the basisof a user recognition result and an environment recognition result bythe spatial information acquisition unit 102. Note that context of acommunication source space may include a state of a user, a state, time,a season, weather, or a place of a real space where the user is, or arelationship with a partner user.

The spatial distance control unit 104 has a function of controlling athree-dimensional spatial distance (a separation distance in a depthdirection) between connected spaces. In the present embodiment,information acquired from each communication destination space (e.g.,the house object 21 into which the video 22 is fitted illustrated inFIG. 3) is placed at an appropriate three-dimensional spatial distanceto show depth in the connection, which enables a comfortable connectionstate. It is assumed that the user's psychological reluctance decreasesas the distance to the partner space becomes longer, and the user'spsychological reluctance increases as the distance to the partner spacebecomes shorter. That is, a short distance between spaces causes partnervideo to look clear, and the spaces are connected with a high sense ofrealism. On the other hand, a long distance between spaces causespartner video to look unclear, and the spaces are connected with a lowsense of realism.

The distance to the partner space may be controlled in accordance with adegree of exposure calculated on the basis of a determination result(context of a communication source space) output from the statedetermination unit 103, for example, or may be controlled on the basisof a degree of exposure manually set by the user. For example, thespatial distance control unit 104 makes the distance to the partnerspace longer in the case where the user is doing housework, movingbusily, or relaxing while watching TV. In addition, for example, thespatial distance control unit 104 may make the distance to the partnerspace shorter in the case where the user approaches the display 1091.The distance to the partner space is transmitted from the communicationunit 107 to each communication-destination information processing device10 and synchronized, as a degree of exposure. The degree of exposure maybe adjusted by the information processing device 10 or the processingserver 30. For example, the degree of exposure may be adjusted to anoptimum degree of exposure on the basis of a degree of exposure desiredin the user-side space and a degree of exposure desired in the partnerspace.

The operation interface (I/F) 105 accepts operation input from a user,and outputs the operation input to the spatial distance control unit 104and the 3D space generation unit 106. This enables, for example, theuser to freely set “a distance to a partner space”, and also enablesoperation input to be performed on various objects (virtual objects)arranged in the three-dimensional space.

The 3D space generation unit 106 generates a 3D space to be displayed onthe display 1091. Specifically, for example, the 3D space generationunit 106 generates a 3D spatial image in which houses corresponding tocommunication destination spaces are arranged, in accordance with adistance to each communication partner set by the spatial distancecontrol unit 104. In addition, the 3D space generation unit 106 mayreflect information from the communication source or the communicationdestination in the 3D space in real time, or arrange designated objects(virtual objects). In addition, scenes such as the background of the 3Dspace may be applied variously by automatic setting or user setting. Forexample, various scenes, such as in a forest, in a city, in space, andunder the sea, can be set.

In addition, the 3D space generation unit 106 according to the presentembodiment can visualize a communication state between communicationpartners in a 3D space. In continuous connection with a plurality ofspaces, visualizing a communication state between participants otherthan the user makes it possible to find the atmosphere of communicationthat has previously occurred; thus, the user can appropriately determinewhether or not to participate in the communication. For example, even ina state where houses of a plurality of partners are arranged remotely,circumstances are unclear, and conversation cannot be heard well,visualization of the communication state enables the situation to beappropriately determined. This prevents occurrence of problems incommunication, and on the contrary enables active participation incommunication, and as a result can enable comfortable multipointcontinuous connection. For example, determination can be made, such asdeciding not to participate when finding that one's mother and youngersister are talking in a gloomy atmosphere for some reason, or decidingto participate for only a short time when being a little busy butfinding that one's mother and younger brother are having a conversationjoyfully.

Communication is visualized in accordance with, for example, activity(excitement, density of conversation, development of conversation, etc.)or a mood (emotion: happiness, anger, sadness, and joy) of conversationbetween communication partners. The 3D space generation unit 106calculates the activity or the mood of the conversation on the basis ofvoice and video of conversation transmitted from each communicationdestination via the communication unit 107. For example, the 3D spacegeneration unit 106 calculates the activity of the conversation fromvolume of the voice, duration of the voice, the number of exchangedwords, a size and frequency of gesture and nodding of the speakingperson, a distance from the display 1091, the number of persons facingthe display 1091, a speed and frequency of development of theconversation, or the like. In addition, the 3D space generation unit 106estimates the mood from volume of the voice, duration of the voice, aspeed of speech, words and wording that are used, strength of the toneof voice, a facial expression, a line of sight (turning one's eyes awayor staring), a size and frequency of gesture of the speaking person, orthe like. An algorithm for calculating the activity and an algorithm forestimating the mood are not particularly limited, and an existingalgorithm may be used. The 3D space generation unit 106 performsvisualization expression in accordance with the calculated activity andthe estimated mood.

Note that a communication state (activity of conversation, a mood ofconversation) is not limited to being acquired by the 3D spacegeneration unit 106 analyzing received data (sound and video) from acommunication partner, and a result of analysis on the communicationpartner side or the processing server 30 side may be received.

The communication unit 107 connects to another information processingdevice 10 or the processing server 30 via the network 20, and transmitsand receives data. For example, the communication unit 107 transmitsspatial information output from the spatial information acquisition unit102, a spatial distance output from the spatial distance control unit104, and information of a 3D space output from the 3D space generationunit 106 to the communication-destination information processing device10 or processing server 30. In addition, the communication unit 107receives spatial information, a spatial distance, information of a 3Dspace, etc. received from the communication-destination informationprocessing device 10 or processing server 30. In the present embodiment,at the communication source and the communication destination, a 3Dspace and a three-dimensional spatial distance to the partner aresynchronized. In addition, the communication unit 107 can receiveinformation (weather information, news, schedule information, etc.)acquired by the processing server 30 from a related service server on anetwork, or directly receive the information from the related serviceserver on the network.

The spatial information generation unit 108 generates spatialinformation on the basis of a 3D spatial image generated by the 3D spacegeneration unit 106 and video of a communication destination spacereceived via the communication unit 107. The generated spatialinformation is sent to the output unit 109. Specifically, the spatialinformation generation unit 108 generates spatial image information bycombining video of a communication destination space received via thecommunication unit 107 to a virtual house image of a 3D spatial imagegenerated by the 3D space generation unit 106. In addition, the spatialinformation generation unit 108 generates spatial audio information onthe basis of sound that occurs in the generated 3D space and sound of acommunication destination space received via the communication unit 107.At this time, spatial audio information may be information obtained byperforming auditory lateralization on communication destination sound inaccordance with arrangement of houses in the 3D space.

The output unit 109 functions as an output control unit that performscontrol so as to output spatial information generated by the spatialinformation generation unit 108 from each device to the user. Forexample, the output unit 109 includes the display 1091, the speaker1092, or the indicator 1093 (an LED lamp etc.), and performs displayoutput, audio output, and notification output. On the display 1091 isdisplayed spatial image information generated by the spatial informationgeneration unit 108. In addition, on the speaker 1092 is displayedspatial audio information generated by the spatial informationgeneration unit 108.

The storage unit 110 stores data transmitted and received via thecommunication unit 107. In addition, the storage unit 110 is connectedto the communication unit 107 in the example illustrated in FIG. 4, butthe present embodiment is not limited to this, and the storage unit 110may be used as a storage area of each component included in theinformation processing device 10.

The configuration of the information processing device 10 according tothe present embodiment has been specifically described. Note that theconfiguration of the information processing device 10 is not limited tothe example illustrated in FIG. 4; for example, part of theconfiguration may be implemented by an external device, and may beconnected to the information processing device 10 in a wired/wirelessmanner.

3. OPERATION PROCESSING <3-1. Communication Control Processing>

FIG. 5 is a flowchart for describing communication control processingaccording to the present embodiment. Operation processing illustrated inFIG. 5 is similarly performed in each of spaces A to C that areconnected, but operation processing of the information processing device10A in space A is described here as an example.

As illustrated in FIG. 5, first, the information processing device 10Aperforms connection processing on space B and space C (step S103).Specifically, for example, the information processing device 10A makes acall to the information processing devices 10B and 10C and establishes asession in the case where permission for connection is received from thepartners.

Next, the information processing device 10A acquires spatial information(video, sound, sensor data) of space A by the spatial informationacquisition unit 102 (step S106).

Then, the information processing device 10A determines a state of user Aand a state of space A by the state determination unit 103 (step S109).State information may be manually input by user A from the operation I/F105.

Next, the information processing device 10A transmits spatialinformation and state information to the information processing device10B in space B and the information processing device 10C in space C thatare connected by communication (step S112).

Then, the information processing device 10A receives real-time spatialinformation and state information of space B and space C similarlyacquired in space B and space C from the information processing device10B and the information processing device 10C (step S115).

Next, the information processing device 10A generates a 3D space by the3D space generation unit 106 (step S118). For example, the 3D spacegeneration unit 106 generates a 3D space in which virtual house objectscorresponding to respective spaces are arranged, in accordance withdistances controlled by the spatial distance control unit 104. Inaddition, the 3D space generation unit 106 visualizes a communicationstate in real time on the basis of conversation voice and video betweencommunication partners. Communication state visualization processingwill be described next with reference to FIG. 6.

Then, the information processing device 10A outputs spatial informationgenerated by the spatial information generation unit 108 from the outputunit 109. The spatial information generation unit 108 generates spatialimage information obtained by combining real-time video of space B andspace C received via the communication unit 107 with the respectivehouse objects in the 3D space generated by the 3D space generation unit106, for example, and outputs the spatial image information to thedisplay 1091.

<3-2. Visualization Processing>

FIG. 6 is a flowchart illustrating communication state visualizationprocessing according to the present embodiment. As illustrated in FIG.6, first, the 3D space generation unit 106 of the information processingdevice 10A analyzes conversation voice and video of each communicationpartner (step S203).

Next, the 3D space generation unit 106 calculates activity ofconversation (step S206) and estimates a mood of conversation (stepS209), as a communication state.

Then, the 3D space generation unit 106 visualizes the communicationstate in a 3D space in accordance with the calculated activity ofconversation or the mood of conversation (step S212). Specific examplesof visualization expression will be described later with reference toFIGS. 12 to 27.

The operation processing according to the present embodiment has beenspecifically described. Now, specific display examples of spatialinformation according to the present embodiment will be described usingdrawings.

4. DISPLAY EXAMPLES OF SPATIAL INFORMATION <4-1. Medium-DistanceDisplay>

First, medium-distance display with a “medium” degree of exposure isdescribed with reference to FIG. 7. FIG. 7 illustrates a display exampleof a 3D space in which house objects are arranged at a medium distanceaccording to the present embodiment. In the illustrated example, awindow frame image 40 and a 3D spatial image 41 seen via a window aredisplayed on the display 1091 that is large-sized and installed on awall or the like.

In the 3D spatial image 41, the house object 21B corresponding to spaceB and the house object 21C corresponding to space C that are connectedby communication are arranged. In the present embodiment, a distancefrom the window frame image 40 to the house object 21B and the houseobject 21C as illustrated in FIG. 7 as an example may be defined as“medium distance”, and this may be set as a usual state. A usual stateis a normal state in continuous connection, and the informationprocessing device 10 constantly performs control at a medium distanceunless there is some sort of trigger. The “medium distance” is adistance in which video is visible enough for circumstances of thepartner space to be roughly grasped, does not place a largepsychological load, and is a sense of distance that enables each user tofind the circumstances of the partner without being annoyed by the userhim/herself being seen. In addition, in the case of “medium distance”,the information processing device 10A controls volume of the sound ofthe partner space to moderate volume. The “medium distance” illustratedin FIG. 7 is an example: “medium distance” may be set in advance, may befreely input and adjusted by each user at the beginning of connection,or may be adjusted as appropriate on the system side on the basis of astate of each user.

Note that in the case of continuous connection at “medium distance”, asituation in which one does not want to be seen by the partner mayoccur. In this case, in order to secure the user's privacy, theinformation processing device 10 can make video difficult to see fromthe partner, and also perform display that enables the user tointuitively find that the video is difficult to see for the partner. Inthe example illustrated in FIG. 7, a virtual window of the house object21C is expressed by frosted glass, and the video 22C of space C is in ablurry state. This is because blocking is freely controlled by user C ofspace C.

In addition, another example of privacy protection is described withreference to FIG. 8. FIG. 8 illustrates an example of privacy protectionaccording to the present embodiment. As illustrated in FIG. 8, forexample, a blind image 42 is displayed on the display 1091 to enable theuser to intuitively grasp that video is difficult to see from thepartner. In this case, as a matter of course, a blind image is similarlydisplayed superimposed on video of space A that is fitted into the houseobject corresponding to the own space placed in the 3D space displayedon the display 1091 in the partner space, and is synchronized.

How much the blind is drawn and its appearance may be controlled by useroperation (e.g., touch operation on the display 1091). Thus virtuallydrawing the blind also serves as indication of one's intention of notwanting to be seen or talked to at present.

Note that the example of the blind illustrated in FIG. 8 is an exampleof a method for expressing privacy protection (blocking), and there alsomay be expression methods such as providing a window screen on thedisplay 1091 and closing a curtain, for example. In addition, in thecase where the display 1091 is likened to a virtual door, expressionsuch as closing the door a little is also possible.

<4-2. Short-Distance Display>

Next, short-distance display with a “high” degree of exposure isdescribed with reference to FIG. 9. FIG. 9 illustrates a display exampleof a 3D space in which house objects are arranged at a short distanceaccording to the present embodiment.

In the present embodiment, a distance from the window frame image 40 tothe house object 21B and the house object 21C as illustrated in FIG. 9as an example may be defined as a range of “short distance”, and thismay be set as a state where conversation is possible. As illustrated inFIG. 9, at the “short distance”, video can be seen clearly enough forthe circumstances of the partner space to be sufficiently grasped. Inaddition, the information processing device 10 controls output so thatsound of the partner space can be sufficiently heard, to enableconversation. Distances to the house objects 21 can be controlledindividually (for example, it is possible to approach only the houseobject 21B).

In addition, at the “short distance”, the stage 24 installed between thehouse objects can be used for many people to work together and exchangeinformation. For example, as illustrated in FIG. 9, a virtual pet object25 can be displayed, and it can be taken care of by feeding inaccordance with user operation. For example, as illustrated in FIG. 9,user A of space A can give feed 26A by operation, and user B of space Bcan similarly give feed 26B by operation. Thus, mutual communication canbe enabled via an object displayed in the 3D space. That is, influencecan be exerted from each space on various objects arranged in the 3Dspace, which enables communication through the objects.

Such use of the 3D space is not limited to the example illustrated inFIG. 9, and there may be other various examples. Specific examples aregiven below.

-   -   Virtual flowers and plants can be arranged in a 3D space, and        users can grow them together by watering them.    -   A bay window may be displayed, and virtual plants may be        arranged at the bay window. Users can buy and arrange virtual        plants, or present them to a partner.    -   A fireplace may be placed at the center of a 3D space, and        flames may be caused in the case where a user throws in        firewood.    -   A specific object extracted from video of a partner space may be        placed in a 3D space. For example, only a child appearing in the        video of the partner space may be extracted and placed in the 3D        space. This makes it possible to show only the child playing in        a room even in the case where the room is messy and one does not        really want the room to be seen.

In addition, in the 3D space, information from each space may bereflected in real time. For example, illumination (color temperature,brightness, fluctuations), room temperature, and environmentalinformation (weather etc.) in each space may be reflected in the 3Dspace. In the case of reflecting the weather of each space, for example,in the case where it is actually raining in a region of space B, rainmay be caused to fall around the house object 21B of space B placed inthe 3D space. This enables each user to grasp a weather situation of thepartner space in real time.

In addition, in the 3D space, a virtual bulletin board on which userscan freely write may be set up. In addition, on the virtual bulletinboard, an abridgment of one day may be automatically displayed on thebasis of the users' action information of the day, for example.

<4-3. Zero-Distance Display>

In addition, further approach from the short distance results in a statewhere virtual windows are connected, that is, zero-distance display. Inthis case, clearer video may be displayed. FIG. 10 illustrates anexample of zero-distance display.

As illustrated on the left of FIG. 10, further approach to the partnerhouse objects 21B and 21C causes virtual windows to be connected,resulting in zero-distance display as illustrated on the right of FIG.10. In zero-distance display, for example, the video 22B of space B andthe video 22C of space C are displayed side by side. In this case, soundand information of the 3D space are blocked, which enables concentrationon communication.

<4-4. Long-Distance Display>

Next, long-distance display with a “low” degree of exposure is describedwith reference to FIG. 11. FIG. 11 illustrates a display example of a 3Dspace in which house objects are arranged at a long distance accordingto the present embodiment.

As illustrated in FIG. 11, in the case of willing to go away from thepartner space, for example, the user can freely perform operation ofsetting the degree of exposure to “low” to go far away from the houseobjects 21B and 21C. In this case, as a matter of course, a distance(degree of exposure) is synchronized so that the house objectcorresponding to the own space placed in the 3D space displayed on thedisplay 1091 in the partner space is displayed remotely.

<4-5. Visualization of Communication State>

Now, specific examples of visualization of a communication stateaccording to the present embodiment will be described. As describedabove, in the present embodiment, in the case where communication hasalready occurred between communication partners (in the case where aconversation is being made), visualizing a state of such communicationenables the user to appropriately determine whether or not to activelyparticipate in the communication.

FIG. 12 illustrates an example in which occurrence of communication isvisualized according to the present embodiment. As illustrated in FIG.12, in the case where the house objects 21B and 21C are placed at ashort distance, for example, in the 3D space displayed on the display1091 and communication occurs between the house objects 21B and 21C, theinformation processing device 10 displays visualization display 50connecting virtual windows of the house objects. The visualizationdisplay 50 is expressed in a simple line form, for example. This enablesthe user to, when seeing the display 1091, intuitively grasp thatcommunication has already occurred between the communication partners(user B and user C). Hereinafter, variations of such visualizationexpression will be described using a plurality of specific examples.

(4-5-1. Types of Visualization Expression)

FIGS. 13 to 15 are diagrams for describing types of visualizationexpression according to the present embodiment. The informationprocessing device 10 does not perform visualization display in a statewhere no communication has occurred between partner spaces, asillustrated on the left of FIG. 13, and performs display of connectingvirtual windows of the house objects 21B and 21C by visualizationdisplay 51 expressing the activity of conversation with a waveform-likedesign, for example, in a state where communication has occurred betweenpartner spaces, as illustrated on the right of FIG. 13.

In addition, the information processing device 10 may perform display offloating visualization display 52 expressing the activity ofconversation with objects like speech balloons of comics, for example,in the sky above the house objects 21B and 21C in a state wherecommunication has occurred between partner spaces, as illustrated on theleft of FIG. 14. In addition, the information processing device 10 maydisplay visualization display 53 with an expression as if virtualwindows of the house objects 21B and 21C are connected in a state wherecommunication has occurred between partner spaces, as illustrated on theright of FIG. 14.

In addition, the information processing device 10 may displayvisualization display 54 with an expression by simple connection with aline, for example, in a state where communication has occurred betweenpartner spaces, as illustrated on the left of FIG. 15, or performdisplay of connecting virtual windows by visualization display 55further expressing the activity of conversation with a broken line, asillustrated on the right of FIG. 15.

As described above, visualization expression of a communication stateaccording to the present embodiment includes various types, such as asimple line, a broken line, a wavy line, or a speech balloon. In all thecases, occurrence of communication between partner spaces is expressedby using expression connecting the partner spaces.

Now, specific examples will be given below in regard to a method forexpressing activity of conversation or a mood of conversation for eachtype of such visualization expression.

(4-5-2. Expression of Activity of Conversation)

FIGS. 16 to 20 are diagrams for describing methods for expressingactivity of conversation according to the present embodiment.

As illustrated in FIG. 16, in the case of using the visualizationdisplay 51 using a waveform, for example, the activity of conversationcan be expressed by intensity of waves. As illustrated in the drawing,the information processing device 10 displays visualization display 51-1with a gentle waveform in the case where the activity of conversation iscalm, displays visualization display 51-2 with a slightly intensewaveform when the activity of conversation becomes slightly active, anddisplays visualization display 51-3 with an intense waveform when theactivity of conversation becomes active. This enables the user tointuitively grasp the activity of conversation.

In addition, as illustrated in FIG. 17, in the case of using thevisualization display 52 using speech balloons of comics, for example,the activity of conversation can be expressed by the size and number ofspeech balloons. As illustrated in the drawing, the informationprocessing device 10 displays visualization display 52-1 with a smallnumber of small speech balloons in the case where the activity ofconversation is calm, displays visualization display 52-2 with anincreased number of medium-sized speech balloons when the activity ofconversation becomes slightly active, and displays visualization display52-3 with an increased number of speech balloons including large speechballoons when the activity of conversation becomes active. This enablesthe user to intuitively grasp the activity of conversation.

In addition, as illustrated in FIG. 18, in the case of using thevisualization display 53 connecting virtual windows, for example, theactivity of conversation can be expressed by darkness of color of thevisualization display 53. As illustrated in the drawing, the informationprocessing device 10 displays visualization display 53-1 with a lightcolor in the case where the activity of conversation is calm, displaysvisualization display 53-2 with a color of medium darkness when theactivity of conversation becomes slightly active, and displaysvisualization display 53-3 with a dark color when the activity ofconversation becomes active. This enables the user to intuitively graspthe activity of conversation.

In addition, as illustrated in FIG. 19, in the case of using thevisualization display 55 using a broken line, for example, the activityof conversation can be expressed by density of the broken line. Asillustrated in the drawing, the information processing device 10displays visualization display 55-1 with a reduced density of the brokenline in the case where the activity of conversation is calm, displaysvisualization display 55-2 with a slightly increased density of thebroken line when the activity of conversation becomes slightly active,and displays visualization display 55-3 with an increased density of thebroken line when the activity of conversation becomes active. Thisenables the user to intuitively grasp the activity of conversation.

In addition, as illustrated in FIG. 20, in the case of usingvisualization display 56 lighting surroundings of virtual windows in asimilar way, for example, the activity of conversation can be expressedby strength of light. As illustrated in the drawing, the informationprocessing device 10 displays visualization display 56-1 weakly lightingsurroundings of a virtual window of each partner space in the case wherethe activity of conversation is calm, displays visualization display56-2 slightly strongly lighting the surroundings when the activity ofconversation becomes slightly active, and displays visualization display56-3 strongly lighting the surroundings when the activity ofconversation becomes active. This enables the user to intuitively graspthe activity of conversation.

As described above, how active (intense) the activity of conversation iscan be expressed by intensity of a wavy line, density of a broken line,the size and number of speech balloons, darkness of color, strength oflight, etc.

(4-5-3. Expression of Mood of Conversation)

Now, a method for expressing a mood of conversation will be described.FIGS. 21 and 22 are diagrams for describing methods for expressing amood of conversation according to the present embodiment.

In the case of using the visualization display 51 using a waveform, forexample, the information processing device 10 displays visualizationdisplay 51 a with a (smooth) waveform having curves and low peaksillustrated in the upper stage of FIG. 21 in the case where the mood ofconversation is gentle, displays visualization display 51 b with anacute waveform illustrated in the middle stage of FIG. 21 in the casewhere the mood of conversation is harsh, and displays visualizationdisplay 51 c with a (rather intense) waveform having curves and highpeaks illustrated in the lower stage of FIG. 21 in the case where themood of conversation seems joyful. In addition, the background color (animage of a color, such as a cold color or a warm color) of the waveformmay further be set to a color corresponding to the mood. This enablesthe user to intuitively grasp the mood of conversation.

Note that the information processing device 10 can additionally includeexpression of the activity of conversation by intensity of a wavy linedescribed above, and express both the activity of conversation and themood of conversation, such as being active conversation in a harsh moodor being calm conversation in a joyful mood, for example.

In addition, in the case of using the visualization display 52 usingspeech balloons, for example, the information processing device 10displays visualization display 52 a using circular speech balloonsillustrated in the upper stage of FIG. 22 in the case where the mood ofconversation is gentle, displays visualization display 52 b of explosivespeech balloons illustrated in the middle stage of FIG. 22 in the casewhere the mood of conversation is harsh, and displays star-shapedvisualization display 52 c illustrated in the lower stage of FIG. 22 inthe case where the mood of conversation seems joyful. This enables theuser to intuitively grasp the mood of conversation.

(4-5-4. Participation Rejection Display)

In addition, in the communication system 1 for multi-person continuousconnection, a situation in which “one does not want a specific person toparticipate” may occur. In the present embodiment, participants thathave previously started communication can clearly display “participationrejection” in the 3D space, to indicate one's intention of rejectingjoining of a third person or not wanting a third person to participate.

The display of “participation rejection” can be actively selected andcanceled by a user. In addition, the system side can grasp thecommunication state and automatically display “participation rejection”.For example, in the case where a predetermined keyword such as “let'skeep this between us” and “it's a secret” is extracted from theconversation, “participation rejection” rejecting joining of anotherperson may be automatically displayed.

Here, display examples of “participation rejection” are described withreference to FIGS. 23 and 24. FIG. 23 illustrates an example ofparticipation rejection display according to the present embodiment. Inthe illustrated example, the house objects 21B (house objects areillustrated simplified in FIG. 24) and 21C are covered with dome-likeenclosure display 60; thus, it is clearly expressed that user B and userC do not want another person to be involved in their communication.

FIG. 24 illustrates another example of participation rejection displayaccording to the present embodiment. In the illustrated example, fencedisplay 61 is displayed on the front side of the house objects 21B and21C; thus, it is clearly expressed that user B and user C do not wantanother person to be involved in their communication.

FIG. 25 illustrates another example of participation rejection displayaccording to the present embodiment. In the illustrated example, curtaindisplay 62 is displayed on the front side of the house objects 21B and21C, and a prohibition mark 63 is added to the curtain display 62; thus,it is more clearly expressed that user B and user C do not want anotherperson to be involved in their communication.

As described above, according to the present embodiment, displayingparticipation rejection makes it possible to indicate one's intention ofrejecting joining of a third person or not wanting a third person toparticipate. Thus, for example, there may be the following use case: inthe case where the communication system 1 enables multipoint continuousconnection in which a mother and her daughter and son participate,participation rejection is displayed temporarily to the son when thewomen want to talk between them, for example. There also may be thefollowing use case: in the case where the communication system 1 enablesmultipoint continuous connection in which a father away from home forwork, his home, and grandparents participate, participation rejection isdisplayed temporarily to the grandparents when the father wants tolecture his son, for example.

(4-5-5. Participation Request Display)

In addition, in the communication system 1 for multi-person continuousconnection, a situation in which “one wants a specific person toparticipate” may occur. In the present embodiment, participants thathave previously started communication can clearly display “participationrequest” in the 3D space, to request joining of a third person.

The display of “participation request” can be actively selected andcanceled by a user. Displaying the participation request enablesparticipants requested to participate to freely determine whether or notto participate. In addition, this enables participants with lowparticipation rates to participate easily.

Here, display examples of “participation request” are described withreference to FIGS. 26 and 27. FIG. 26 illustrates an example ofparticipation request display according to the present embodiment. Inthe illustrated example, an icon 64 making a beckoning motion isdisplayed in the sky above the house objects 21B and 21C in the 3Dspace; thus, it is clearly expressed that user B and user C want anotherperson to participate in their communication.

FIG. 27 illustrates another example of participation request displayaccording to the present embodiment. In the illustrated example, an icon65 making a motion as if a bell is swaying and ringing is displayed inthe sky above the house objects 21B and 21C; thus, it is clearlyexpressed that user B and user C want another person to participate intheir communication.

As described above, according to the present embodiment, displaying aparticipation request makes it possible to request joining of a thirdperson. Thus, for example, there may be the following use case: in thecase where a mother and her daughter have been previously talking in thecommunication system 1 but an older brother cannot seem to participatein the conversation, a participation request is displayed to the olderbrother.

5. HARDWARE CONFIGURATION

The embodiment of the present disclosure has been described. Theprocessing performed by the information processing device 10 describedabove may be achieved by operating cooperatively software and hardwareof an information processing device 100 (to be described later).

FIG. 28 is an explanatory diagram illustrating a hardware configurationof the information processing device 100 according to the presentdisclosure. As illustrated in FIG. 28, the information processing device100 includes a central processing unit (CPU) 142, read only memory (ROM)144, random access memory (RAM) 146, a bridge 148, a bus 150, aninterface 152, an input device 154, an output device 156, a storagedevice 158, a drive 160, a connection port 162, and a communicationdevice 164.

The CPU 142 functions as an arithmetic processing device and a controldevice to enable operation of the spatial information acquisition unit102, the state determination unit 103, the spatial distance control unit104, the 3D space generation unit 106, and the spatial informationgeneration unit 108 of the information processing device 10 by operatingcooperatively with various kinds of programs. In addition, the CPU 142may be a microprocessor. The ROM 144 stores programs, operationparameters, or the like used by the CPU 142. The RAM 146 transientlystores programs used when the CPU 142 is executed, parameters thatchange as appropriate when the CPU 142 is executed, or the like. The ROM144 and the RAM 146 implement part of the storage unit 17 in theinformation processing system. The CPU 142, the ROM 144, and the RAM 146are connected to each other through an internal bus including a CPU busand the like.

The input device 154 implements the camera 1011, the mic 1012, thesensor 1013, and the operation I/F 1015 of the information processingdevice 10, and includes an input control circuit or the like thatoutputs an input signal to the CPU 142.

The output device 156 implements, for example, the display 1091, thespeaker 1092, and the indicator 1093 of the information processingdevice 10. As examples of the display 1091, a liquid crystal display(LCD) device, an organic light emitting diode (OLED) device, and thelike are assumed.

The storage device 158 is a data storage device that implements, forexample, the storage unit 110 of the information processing device 10.The storage device 158 may include a storage medium, a recording devicethat records data on the storage medium, a reading device that readsdata from the storage medium, a deletion device that deletes datarecorded on the storage medium, and the like. The storage device 158stores various kinds of data or a program to be executed by the CPU 142.

The drive 160 is a reader/writer for a storage medium, and isincorporated in or externally attached to the information processingdevice 100. The drive 160 reads information recorded in a removablestorage medium that is mounted, such as a magnetic disk, an opticaldisc, a magneto-optical disk, or semiconductor memory, and outputs theinformation to the RAM 144. The drive 160 is also capable of writinginformation to the removable storage medium.

The connection port 162 is, for example, a bus used to connect to aninformation processing device or peripheral equipment outside theinformation processing device 100. In addition, the connection port 162may be a universal serial bus (USB).

The communication device 164 is, for example, a communication interfaceimplemented by communication equipment for connection with a network, asan example of the communication unit 107 of the information processingdevice 10. In addition, the communication device 164 may be a devicesupporting infrared communication, a communication device supporting awireless local area network (LAN), a communication device supportinglong term evolution (LTE), or a wired communication device that performswired communication.

6. CONCLUSION

As described above, the information processing system according to theembodiment of the present disclosure can enable more comfortablecommunication by rendering distances to multiple communication partnersby using three-dimensional expression for connection between spaces.

Specifically, a 3D space is displayed on a large-sized display installedon a wall or the like and likened to a window or a doorway, and virtualobjects such as houses into which video of a plurality of communicationpartners is fitted are arranged in the 3D space; thus, a distance to acommunication partner can be visually expressed. Note that an aspectratio of the display installed on a wall surface may be implemented inaccordance with dimensions used in actual windows or doors. In addition,connection between a 3D space and a real space can be expressed withmore reality by placing the display such that its lower side is locatednear a floor surface, and performing display as if being contiguous withthe ground of the 3D space.

In addition, video of a communication partner placed in a 3D space isfitted into a virtual object such as a house and placed, and isdisplayed enlarged/reduced as a distance becomes shorter or longer inthe 3D space; thus, a sense of life-size is not lost.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, a computer program for causing hardware such as a centralprocessing unit (CPU), read only memory (ROM), and random access memory(RAM) built in the information processing device 10 or the processingserver 30 described above to exhibit functions of the informationprocessing device 10 or the processing server 30 can also be produced.Furthermore, a computer-readable storage medium in which the computerprogram is stored is also provided.

In addition, the embodiment described above describes a case where thedisplay 1091 is installed on a wall, but the present embodiment is notlimited to this; a display unit provided in an information processingterminal may be used as the display 1091. In addition, a display unit ofa see-through or non-see-through head mounted display (HMD), forexample, may be used as the display 1091.

Note that in the case where an information processing terminal providedwith a small-sized display, such as a smartphone, is used, display likenormal video chat may be performed by screen division on the small-sizeddisplay, instead of 3D space display as illustrated in FIG. 3.

In addition, the embodiment described above is described using acommunication system in which two spaces of communication partners arearranged in a 3D space and three persons participate, but the presentembodiment is not limited to this; two persons or four or more personsmay participate in the communication system.

In addition, when the user approaches the camera 1011 too much,resulting in zoom-in video, a sense of life-size is lost when the videois fitted into the house object 21. At this time, the communicationsystem 1 may cut out a facial image of the user and combine it with abody image generated from previous video or the like, for example, togenerate video that does not lose a sense of life-size and does not seemunnatural when fitted into the house object 21.

In addition, video of a partner space is fitted into a virtual objectsuch as a house in the embodiment described above, but the presentembodiment is not limited to this; more abstractly, expression as if awindow (video of the partner space) is floating may be used.

In addition, motion capture may be performed, and video in which motionof the partner is displayed as an avatar may be used as video of thepartner space.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing device including:

a reception unit configured to receive data from a communicationdestination;

a generation unit configured to generate spatial image informationplaced at a predetermined position in a three-dimensional space by usingcaptured images of a plurality of the communication destinations; and

a display control unit configured to perform control so as to displaythe generated spatial image information on a display unit.

(2)

The information processing device according to (1), in which thegeneration unit visualizes communication that has occurred between theplurality of communication destinations in the three-dimensional space.

(3)

The information processing device according to (2), in which thegeneration unit visualizes a state of the communication.

(4)

The information processing device according to (3), in which thegeneration unit calculates activity of conversation in thecommunication, and performs visualization display corresponding to theactivity.

(5)

The information processing device according to (4), in which thegeneration unit calculates the activity on the basis of at least one ofconversation voice or video during conversation.

(6)

The information processing device according to (3), in which thegeneration unit estimates a mood of conversation in the communication,and performs visualization display corresponding to the mood.

(7)

The information processing device according to (6), in which thegeneration unit estimates the mood on the basis of at least one ofconversation voice or video during conversation.

(8)

The information processing device according to (3), in which thegeneration unit calculates activity of conversation and estimates a moodof conversation in the communication, and performs visualization displaycorresponding to the activity and the mood.

(9)

The information processing device according to any one of (1) to (8), inwhich the generation unit fits the captured image of the communicationdestination into an opening of a virtual object, and places the virtualobject at the predetermined position in the three-dimensional space.

(10)

The information processing device according to (9), in which thegeneration unit controls a distance to each virtual object placed in thethree-dimensional space in accordance with a degree of exposure.

(11)

An information processing method including, by a processor:

receiving data from a communication destination;

generating spatial image information placed at a predetermined positionin a three-dimensional space by using captured images of a plurality ofthe communication destinations; and

performing control so as to display the generated spatial imageinformation on a display unit.

(12)

A program causing a computer to function as:

a reception unit configured to receive data from a communicationdestination;

a generation unit configured to generate spatial image informationplaced at a predetermined position in a three-dimensional space by usingcaptured images of a plurality of the communication destinations; and

a display control unit configured to perform control so as to displaythe generated spatial image information on a display unit.

REFERENCE SIGNS LIST

-   1 communication system-   10 (10A to 10C) information processing device-   20 network-   30 processing server-   101 input unit-   1011 camera-   1012 mic-   1013 sensor-   102 spatial information acquisition unit-   103 state determination unit-   104 spatial distance control unit-   105 operation I/F-   106 3D space generation unit-   107 communication unit-   108 spatial information generation unit-   109 output unit-   1091 display-   1092 speaker-   1093 indicator-   110 storage unit

1. An information processing device comprising: a reception unitconfigured to receive data from a communication destination; ageneration unit configured to generate spatial image information placedat a predetermined position in a three-dimensional space by usingcaptured images of a plurality of the communication destinations; and adisplay control unit configured to perform control so as to display thegenerated spatial image information on a display unit.
 2. Theinformation processing device according to claim 1, wherein thegeneration unit visualizes communication that has occurred between theplurality of communication destinations in the three-dimensional space.3. The information processing device according to claim 2, wherein thegeneration unit visualizes a state of the communication.
 4. Theinformation processing device according to claim 3, wherein thegeneration unit calculates activity of conversation in thecommunication, and performs visualization display corresponding to theactivity.
 5. The information processing device according to claim 4,wherein the generation unit calculates the activity on a basis of atleast one of conversation voice or video during conversation.
 6. Theinformation processing device according to claim 3, wherein thegeneration unit estimates a mood of conversation in the communication,and performs visualization display corresponding to the mood.
 7. Theinformation processing device according to claim 6, wherein thegeneration unit estimates the mood on a basis of at least one ofconversation voice or video during conversation.
 8. The informationprocessing device according to claim 3, wherein the generation unitcalculates activity of conversation and estimates a mood of conversationin the communication, and performs visualization display correspondingto the activity and the mood.
 9. The information processing deviceaccording to claim 1, wherein the generation unit fits the capturedimage of the communication destination into an opening of a virtualobject, and places the virtual object at the predetermined position inthe three-dimensional space.
 10. The information processing deviceaccording to claim 9, wherein the generation unit controls a distance toeach virtual object placed in the three-dimensional space in accordancewith a degree of exposure.
 11. An information processing methodcomprising, by a processor: receiving data from a communicationdestination; generating spatial image information placed at apredetermined position in a three-dimensional space by using capturedimages of a plurality of the communication destinations; and performingcontrol so as to display the generated spatial image information on adisplay unit.
 12. A program causing a computer to function as: areception unit configured to receive data from a communicationdestination; a generation unit configured to generate spatial imageinformation placed at a predetermined position in a three-dimensionalspace by using captured images of a plurality of the communicationdestinations; and a display control unit configured to perform controlso as to display the generated spatial image information on a displayunit.